gather with rtweet

library(tidyverse)
library(rtweet)

• https://docs.ropensci.org/rtweet/
• Introduction to gathering tweets with rwteet

Search tweets

search_tweets() - search a keyword(s) or hashtag

I recommend limiting the number of tweets returned (n = 1000) for this training. Otherwise you’ll probably hit a rate limit.

bts <- search_tweets("#BTS", n = 5000, include_rts = FALSE)

Downloading [=>---------------------------------------]   4%
Downloading [=>---------------------------------------]   6%
Downloading [==>--------------------------------------]   8%
Downloading [===>-------------------------------------]  10%
Downloading [====>------------------------------------]  12%
Downloading [=====>-----------------------------------]  14%
Downloading [======>----------------------------------]  16%
Downloading [======>----------------------------------]  18%
Downloading [=======>---------------------------------]  20%
Downloading [========>--------------------------------]  22%
Downloading [=========>-------------------------------]  24%
Downloading [==========>------------------------------]  26%
Downloading [==========>------------------------------]  28%
Downloading [===========>-----------------------------]  30%
Downloading [============>----------------------------]  32%
Downloading [=============>---------------------------]  34%
Downloading [==============>--------------------------]  36%
Downloading [===============>-------------------------]  38%
Downloading [===============>-------------------------]  40%
Downloading [================>------------------------]  42%
Downloading [=================>-----------------------]  44%
Downloading [==================>----------------------]  46%
Downloading [===================>---------------------]  48%
Downloading [===================>---------------------]  50%
Downloading [====================>--------------------]  52%
Downloading [=====================>-------------------]  54%
Downloading [======================>------------------]  56%
Downloading [=======================>-----------------]  58%
Downloading [========================>----------------]  60%
Downloading [========================>----------------]  62%
Downloading [=========================>---------------]  64%
Downloading [==========================>--------------]  66%
Downloading [===========================>-------------]  68%
Downloading [============================>------------]  70%
Downloading [=============================>-----------]  72%
Downloading [=============================>-----------]  74%
Downloading [==============================>----------]  76%
Downloading [===============================>---------]  78%
Downloading [================================>--------]  80%
Downloading [=================================>-------]  82%
Downloading [=================================>-------]  84%
Downloading [==================================>------]  86%
Downloading [===================================>-----]  88%
Downloading [====================================>----]  90%
Downloading [=====================================>---]  92%
Downloading [======================================>--]  94%
Downloading [======================================>--]  96%
Downloading [=======================================>-]  98%
Downloading [=========================================] 100%

bts_dynamite <- search_tweets("#BTS dynamite", n = 5000, include_rts = FALSE)

Downloading [=>---------------------------------------]   4%
Downloading [=>---------------------------------------]   6%
Downloading [==>--------------------------------------]   8%
Downloading [===>-------------------------------------]  10%
Downloading [====>------------------------------------]  12%
Downloading [=====>-----------------------------------]  14%
Downloading [======>----------------------------------]  16%
Downloading [======>----------------------------------]  18%
Downloading [=======>---------------------------------]  20%
Downloading [========>--------------------------------]  22%
Downloading [=========>-------------------------------]  24%
Downloading [==========>------------------------------]  26%
Downloading [==========>------------------------------]  28%
Downloading [===========>-----------------------------]  30%
Downloading [============>----------------------------]  32%
Downloading [=============>---------------------------]  34%
Downloading [==============>--------------------------]  36%
Downloading [===============>-------------------------]  38%
Downloading [===============>-------------------------]  40%
Downloading [================>------------------------]  42%
Downloading [=================>-----------------------]  44%
Downloading [==================>----------------------]  46%
Downloading [===================>---------------------]  48%
Downloading [===================>---------------------]  50%
Downloading [====================>--------------------]  52%
Downloading [=====================>-------------------]  54%
Downloading [======================>------------------]  56%
Downloading [=======================>-----------------]  58%
Downloading [========================>----------------]  60%
Downloading [========================>----------------]  62%
Downloading [=========================>---------------]  64%
Downloading [==========================>--------------]  66%
Downloading [===========================>-------------]  68%
Downloading [============================>------------]  70%
Downloading [=============================>-----------]  72%
Downloading [=============================>-----------]  74%
Downloading [==============================>----------]  76%
Downloading [===============================>---------]  78%
Downloading [================================>--------]  80%
Downloading [=================================>-------]  82%
Downloading [=================================>-------]  84%
Downloading [==================================>------]  86%
Downloading [===================================>-----]  88%
Downloading [====================================>----]  90%
Downloading [=====================================>---]  92%
Downloading [======================================>--]  94%
Downloading [======================================>--]  96%
Downloading [=======================================>-]  98%
Downloading [=========================================] 100%

bts

bts_dynamite

get friends

Find all the accounts a user follows

john_little <-  get_friends("john_little")

john_little

Use the retreived user_id to get more information about those accounts.

john_little_data <- lookup_users(john_little$user_id)

john_little_data

get followers

Who is following me? get_followers()

jrl_flw <- get_followers("john_little")

jrl_flw_data <- lookup_users(jrl_flw$user_id)

jrl_flw_data 

timelines

Get the most recent tweets from an account

rg_tmls <- get_timelines("RhiannonGiddens", n = 3200)

min(created_at)	max(created_at)
2015-04-21 13:59:55	2020-10-27 20:49:18

rg_tmls %>% 
  dplyr::filter(created_at > "2016-01-01") %>%  
  dplyr::group_by(screen_name) %>%
  ts_plot("weeks", trim = 1L) +
  ggplot2::geom_point() +
  geom_smooth(se = FALSE, color = "cadetblue") +
  colorblindr::scale_color_OkabeIto() +
  hrbrthemes::theme_ipsum(grid = "Y") +
  ggplot2::theme(
    legend.title = ggplot2::element_blank(),
    legend.position = "bottom", 
    plot.title = ggplot2::element_text(face = "bold")
    ) +
    ggplot2::labs(
    x = NULL, y = NULL,
    title = "Frequency of Twitter statuses",
    subtitle = "Twitter status (tweet) counts aggregated by week from Jan. 2016",
    caption = "Source: Data collected from Twitter's REST API via rtweet"
  )

NA
NA

get_favorites

Get the most recent favorites from a user

rg_faves <- get_favorites("RhiannonGiddens", n = 3000)

rg_faves

Profiles

Search a users’ profiles

gullah <- search_users("#gullah", n = 1000)

Searching for users...
Finished collecting users!

gullah

get trends

What is trendingin a specific location?

# sf <- get_trends("san franciso")
# durham <- get_trends(lat = 36.0, lng = -78.9)
greensboro <- get_trends("greensboro")

greensboro

Location information

Using some additional R libraries, the location information can be geocoded and then visualize.

First, geocode

Use the tidygeocoder package.

# glimpse(rg_tmls)
rg_places <- rg_tmls %>% 
  drop_na(place_name) %>% 
  select(place_name:bbox_coords) %>% 
  distinct() %>% 
  mutate(addr = glue::glue("{place_full_name}, {country}")) %>% 
  tidygeocoder::geocode(addr, method = "osm")

rg_places

Visuzlize

You can create maps in R. Below is one of the easiest, especially if you know ggplot2

rg_places %>% 
  distinct() %>% 
  drop_na(lat) %>% 
  ggplot(aes(long, lat), color="grey99") +
  borders("world") + 
  geom_point(color = "goldenrod") + 
  ggrepel::geom_label_repel(aes(label = place_full_name), 
                            segment.color = "goldenrod", segment.size = 1,
                            color = "navy") + 
  theme_void()

Second location example

Very similar to above. For accounts with “#gullah” in their profile, and that have location information listed, then geocode the location …..

gullah_places <- gullah %>% 
  drop_na(place_name) %>% 
  select(place_name:bbox_coords) %>% 
  filter(country_code == "US")  %>%
  distinct(place_name, place_full_name, country) %>% 
  mutate(addr = glue::glue("{place_full_name}, {country}")) %>% 
  tidygeocoder::geocode(addr, method = "osm")

gullah_places

And visualize on a US map of the lower 48 states.

You can learn more about basic R mapping from our workshop on mapping with R

gullah_places %>% 
  distinct() %>% 
  drop_na(lat) %>% 
  ggplot(aes(long, lat), color="grey99") +
  borders("state") + 
  geom_point(color = "goldenrod") + 
  ggrepel::geom_label_repel(aes(label = place_full_name), 
                            segment.color = "goldenrod", segment.size = 1,
                            color = "navy") + 
  theme_void()